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ABSTRACT 



Within a course at the University of Colorado addressing the 
management of research and development, a comparison of alternate 
asynchronous delivery media was conducted. For the course session on the 
management of intellectual property, asynchronous students were given access 
to the VHS tapes traditionally utilized, plus World Wide Web-based and 
CD-based (digital) lecture alternatives. The objectives of the comparison 
were: to determine student preferences; to ascertain the penetration of the 
computer technology necessary to utilize the alternative digital delivery 
methods; and to understand the usage patterns for the current videotapes to 
improve the design of future delivery systems. Students had two opportunities 
to express their preferences- -a mailed survey administered immediately after 
the session and a less comprehensive follow-up assessment via the 
university's Faculty Course Questionnaire. With the mailed survey, students 
were asked for feedback in three categories : usage factors for VHS 
videotapes; preferences with respect to the digital delivery alternatives; 
and equipment barriers to utilizing the digital alternatives. Based on the 
survey results, approximately 50% of the students lacked convenient access to 
sufficient computer resources to take advantage of these alternate delivery 
mechanisms. (Contains 10 references.) (MES) 
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Abstract: Unrelenting progress in computer and telecommunications technology is creating new 
opportunities and new challenges in distance education. These technologies have the potential to 
free students and instructors from the bonds of time and space. However, the adoption of new 
technologies should not exclude prospective students. The development and evaluation of an 
online alternative to videotaped lectures is presented. Videotape usage factors, feedback 
regarding online alternatives, and barriers to adoption of online lectures are explored. 



Introduction 

Colleges and universities are rushing to implement distance education [Sherron and Boettcher 1997], but are our 
students ready? Utilizing technology is still problematic for some students [Ingebritsen and Flickinger 1998], so 
new delivery methods should be as easy to use as the technologies that they replace. Less than 23 percent of 
households had computer systems in 1993; if the growth trends at the time hold, approximately 30% of households 
will have computers by the turn of the century [U. S. Census Bureau 1993]. This suggests that adoption of 
emerging distance education technologies brings with it the risk of excluding large numbers of potential students. 

The vast majority of Engineering Management students at the University of Colorado participate asynchronously 
via VHS videotapes provided by the Center for Advanced Training in Engineering and Computer Science 
(CATECS). Students return the videotapes at the end of the semester. Courses have low production values, 
referred to by the CATECS Program Director as "candid classroom". Students admitted to the program are 
required to have obtained a minimum of a bachelor's degree in an engineering or physical science discipline, so a 
higher than average student facility with computers and a higher than average rate of computer penetration into 
student households can be assumed. 



Digital Lecture Design 

Within a course addressing the management of research and development (EMEN5300), a comparison of alternate 
asynchronous delivery media was conducted. For the course session on the management of intellectual property 
(Session 12), asynchronous students were given access to the VHS tapes traditionally utilized with EMEN5300, 
plus web'based and CD-based (digital) lecture alternatives. The objectives of the comparison were (1) to determine 
student preferences, (2) to ascertain the penetration of the computer technology necessary to utilize the alternative 
digital delivery methods, and (3) understand the usage patterns for the current video tapes to improve the design of 
future delivery systems. 

Steps were taken in the delivery media comparison to exert experimental controls over the five factors that 
influence the teaching-learning exchange [Heimlich and Norland 1994]. Each of the delivery alternatives featured 
the same instructor and was delivered to the same student population. The session slides, discussion forum and 
course artifacts were also identical for each of the delivery methods. The session was pre-recorded on VHS and the 
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audio was extracted for use with the digital media, so the lecture audio content was identical for each delivery 
vehicle. However, video content posed some interesting trade-offs. 

Exclusion of the video from the digital media might cause the students feel as though they were "missing 
something" and bias them against the digital alternatives. However, inclusion of the video greatly increased the 
size of the digital files, requiring multiple CD volumes for a two-and-a-half hour lecture, and perhaps slowing 
content delivery to unacceptable levels. A review of the course video revealed that over 50% of the video content 
consisted of images of the session slides. These slides were to be available in another multimedia window in the 
digital delivery environments. Also, with all of the currently available streaming media encoding technology, the 
resolution of these slide video images would render them illegible. The prospect of over 50% of the video content 
being both redundant and illegible also presented the possibility of student bias against the digital delivery 
alternatives. The remaining video content consisted primarily of "talking head" shots of the instructor. One of the 
most fundamental conflicts in distance learning is whether the content should be adjusted to take advantage of the 
new medium or reproduced as faithfully as possible given the constraints imposed by the new technology [Ben- 
Jacob 1998]. Taking all of these factors into consideration, a decision was reached to exclude the video content 
from the digital delivery alternatives and include a question about the value of the video content in the student 
preference survey. 

Simple navigation was a key design goal for the online lesson. Unlike books and videotapes, there are no widely 
accepted principles for organizing and navigating the multi-dimensional electronic information space [Bevirt 
1996]. Principles for organization and navigation were sought that would be applicable across course sessions and 
across courses. An effort was made to develop an interface that could be deployed via either the web or CD with a 
minimum of modification. Unix shell scripts were developed to automate the most redundant and time-consuming 
tasks. Audio for the CDs consisted of full resolution .wav files. Audio for the web was encoded using VivoActive 
because it permits audio streaming via the Hypertext Transfer Protocol (HTTP) so no special web server software 
or configuration was required. 

The starting point for session navigation is the lesson home page. The lesson home page includes the course 
number and the session number within the course. The page also includes hyperlinks to the outline for the lesson, 
the table of contents for the lesson slides, a link to the first slide of the lesson and links to contact the instructor. 
The cover page also includes hyperlinks to download any viewers or plug-ins required for the handouts and the 
lesson. 

Each lesson is decomposed into slides. Slides are generally intended to be viewed sequentially, but the table of 
contents for each lesson facilitates non-linear traversal of the slides. Each slide includes four areas: the slide image 
in the lower left comer, an icon returning the user to the lesson home page in the upper right comer, a horizontal 
navigation bar in the upper left comer, and a vertical tool bar in the lower right comer. The slide image presents 
the unique material to be covered at this point in the lesson, while the home page icon, navigation bar and tool bar 
are intended to be consistent across slides, across lessons, and across courses. The navigation bar and tool bar are 
orthogonal [MacLennan 1 983] in the sense that the navigation bar represents movement within the current lesson 
and the tool bar represents resources outside of the current lesson. The Hypertext Markup Language (HTML) 
"ALT" and "ONMOUSEOVER" tags are utilized throughout the lesson, so that the buttons are self-documenting. 

The navigation bar represents an ordinal scale [Agresti 1990]. The left-hand icon takes the student to an absolute 
location, the first slide in the lesson. Moving to the right, the second icon takes the student to a relative location, 
the slide immediately preceding the current one. The middle icon on the navigation bar provides the “origin” for 
the ordinal scale; it takes the student to another dimension composed of additional multimedia resources that 
elaborate on the current slide. In most cases, the middle icon takes the student to the relevant audio clip. The 
session topic was intellectual property management, so the middle icon sometimes linked to another web site with 
more detailed information, such as the US Patent and Trademark Office or the Library of Congress. The 
remaining two icons on the navigation bar are complementary to the first two icons. The fourth icon takes the 
student to the relative position of the slide immediately following the current one. The last icon on the navigation 
bar takes the student to the absolute position of the last slide in the lesson. 
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The top icon on the tool bar permits the student to contact the instructor via e*mail. Moving down, the second icon 
provides the student access to the threaded course discussion list (HyperNews). The third icon on the tool bar 
guides the student to the home page for the University of Colorado library system. Once there, the student can 
search for literature by title, author, subject and keyword. The fourth icon allows the student to search all web 
pages in the colorado.edu domain, including the library system. The bottom icon allows the student to search the 
entire Internet via the "hotbot" search engine. 



Delivery Technology Assessment 

Students had two opportunities to express their preferences, A mailed survey was administered immediately after 
Session 12, This survey featured nominal and interval response scales, plus the opportunity for open-ended 
response. The mailed survey was reviewed and approved by the University of Colorado Human Subjects 
Committee. After course completion, students had the opportunity for a less comprehensive follow-up assessment 
via the university’s Faculty Course Questionnaire (FCQ). The FCQ utilizes 5 point Likert response scales from 
zero (very poor) to 4 (very good). The FCQ also invites open-ended remarks under four headings: most effective 
aspects, least effective aspects, best ways to improve, and other comments. Participation in both of the surveys was 
voluntary and responses were anonymous. Response rates were much higher for the FCQ than for the mailed 
survey; possibly because the FCQ is already familiar to the students and is administered by the university instead of 
the instructor which may, in turn, inspire more confidence that their anonymity will be protected. 

With the mailed survey, students were asked for feedback in three categories: usage factors for VHS videotapes, 
preferences with respect to the digital delivery alternatives, and equipment barriers to utilizing the digital 
alternatives. 



Videotape Usage Factors 

All nine students responding to the mailed survey, from a total enrollment of 20, indicated that home was the 
primary location for videotape viewing. One student’s open-ended remark indicated that they occasionally viewed 
videotaped lectures during business travel. Seven students indicated that they typically viewed the tapes in one 
sitting and rarely, if ever, rewound to review. One student indicated that they typically watched in one sitting, but 
rewound often. The remaining student indicated that they typically watched a single course session in multiple 
sittings. Two of nine students responding agreed with the idea that they reviewed the videotapes after the initial 
viewing to prepare for homework and/or tests, while the other seven disagreed with that notion. Three respondents 
indicated that they would rather be live on campus for the initial lecture delivery, while 6 stated a preference for 
videotape, and none favored either of the digital alternatives. For reviewing lectures, 3 students preferred 
videotape, 4 preferred the CD and two did not respond to the question. One of the unresponsive students made the 
open-ended remark that they preferred to review from the handouts, an alternative not provided on the multiple- 
choice question. Two students indicated that the audio and video quality of the VHS tapes was occasionally 
unacceptable. Four students indicated that they would value access to the lecture beyond the end of the semester, 
but the other five respondents did not attach any value to extended access to the lectures. It is not clear whether 
students currently enrolled in a course have the proper perspective to know whether they would want access to the 
materials at some point in the future. 



Digital Lecture Preferences 

The number of survey responses regarding the digital alternatives was typically smaller because some students 
lacked sufficient computer resources to fully utilize the web- and CD-based alternatives. Eight out of nine students 
found navigation in the digital lessons simple and intuitive, while one found it to be complex. Six of seven 
respondents said that the slides in the digital presentations had the right balance of legible size and fast loading, 
while one student indicated that the slides needed to be larger. Six of seven also indicated that the opportunity to 



embed links to other web sites was valuable, though one respondent stated that there was no additional value to the 
links. Five out of seven students indicated that the synchronizing aspects of web delivery had the potential to make 
team projects easier to accomplish. 

Three out of six students found that video was not necessary with the digital alternatives, but the other half of the 
responses indicated that video should be included. One student indicated in the open-ended comments that the 
student should be given the flexibility to choose small or large slides and audio-only or audio/video presentation. It 
is believed that if the students had the chance to observe the digital video resolution and recognized the fact that 
the majority of the video content consisted of slide images, that the students would ascribe less value to having the 
video available in the digital format. Fifty percent of the respondents also indicated that the digital audio 
resolution was satisfactory, while the other half stated that higher resolution was preferred. The fact that the 
digital audio was captured from the original VHS recording resulted in degraded digital audio quality. Subsequent 
investigation indicates that direct digital audio capture results in much better audio quality without increasing the 
size of the audio files. 

One student indicated that, as long as they had to be at the computer to view the lecture, an on online editing 
capability for the handouts should also be provided. Another respondent indicated that the middle button on the 
navigation bar should consistently play the audio instead of occasionally leading to other web sites, and that an 
additional button should appear on the navigation bar when, and only when, there are links to additional 
information. With the suggested additional button, the navigation bar would no longer represent an ordinal scale; 
this would make the interface potentially less intuitive. Also, having a button that doesn't appear on every page 
violates interface "consistency" [MacLennan 1983]. On the other hand, a button that plays an audio clip on some 
occasions and links to another web site on others is also a "consistency" violation [ibid.]. Additional testing 
focused of this interface design trade-off is indicated. 

On the FCQ, 17 respondents gave the videotapes an average rating of 1.8 and 13 respondents gave digital delivery 
an average rating of 2.9. The average rating for navigation in the digital lessons was 3.0 for eleven students 
responding. There were four open-ended comments related to the digital delivery alternatives. The comment "I 
liked the CD lecture too; very easy to get to the needed audio portion of a particular lecture" was included as one of 
the most effective aspects of the course. A second respondent also indicated that the CD was one of the most 
effective aspects of the course. "Web presentation ok" was included in the "other comments" section. One student 
indicated "Sessions on CD" among the best ways to improve the course, but it isn't clear whether they are 
indicating a need to improve the CD method of delivery or a desire to have the whole course delivered via CD. 
Under least effective aspects of the course, one student indicated that they could not use the CD, but weren’t sure 
whether the CD or their computer was the source of the problem. Two respondents indicated under "other 
comments" their appreciation for the instructor's creativity in utilizing new tools for distance learning, but a third 
student indicated that the distance education "experimentation" was a source of distraction from the course 
materials. There were three comments indicating quality and/or delivery problems with the VHS videotapes. 



Barriers to Digital Lectures 

In the mailed survey, students were asked to indicate ^ factors that presented barriers to utilizing digital lectures. 
Subsequently, some students identified more than one barrier. Out of nine respondents, there were four indications 
of no sound capabilities, one indication that the computer was too slow, one indication that the CD player was too 
slow, and one indication that the Internet connection was too slow. In the open-ended remarks, three respondents 
indicated that they had computers at work that were capable of utilizing the digital lectures, but that their home 
systems were inadequate. One student indicated plans to buy a sound card. Another indicated that they typically 
didn't have Internet access during occasions of business travel. One student mentioned that they had to click "OK" 
in several dialog boxes each time they played an audio clip from the CD. At least one of these dialog boxes is 
probably associated with the browser; the most popular browsers prompt the user whether they want to save the file 
to disk or "view" the file with the appropriate application. It is possible to configure these browsers so that they 
don't prompt the user each time they access a new file, so this shortcoming may well be addressed by additional 
training. Virus protection software may be the source of another prompt. Norton Anti-Virus can be configured to 
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limit the file types scanned based on the filename extension, so additional training may resolve this deficiency as 
well. If "several'' dialog boxes implied more than two, it is not readily apparent what are the sources of the other 
prompts; the anonymous nature of the survey precluded further investigation of the student's situation. 



Conclusions 

Caution is recommended with respect to a shift of course delivery media to CD or the web. Based on the survey 
results, approximately 50% of the students enrolled in EMEN5300 lacked convenient access to sufficient computer 
resources to take advantage of these alternate delivery mechanisms. Dictating the need for sufficient computer 
resources as a course prerequisite is one way to address this issue, but the high cost of such a requirement could 
jeopardize enrollment levels. However, this situation bears close monitoring since Moore's law indicates that 
computer capabilities double in a mere 18 months [Ziems 1996], 

The concern raised by one student regarding of the issue of distance education "experimentation" is a valid one. 
Many studies, for example [Barker 1998] and [Ingebritsen and Flickinger 1998], compare a traditional course 
taught in one semester or to one group with a revised course taught to another. Although such an approach makes 
it less obvious to the students that they are experimental "subjects", it is still likely that time is being taken away 
from course content to deal with the technology. Such an approach also makes it harder to control experimental 
variables and validate conclusions. In either case, some sort of risk/reward analysis is justified to make sure that 
the benefits of the research warrant the reduction in course content that the experiment may entail. It is also 
recommended that, unless it violates the research protocol, students should be informed of their role in the 
experiment and made aware of the potential benefits for future students. 
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